Skip to content

Pin llama.cpp 45c1770: HRX fuses ZAYA's residual scale - #230

Merged
bong-water-water-bong merged 3 commits into
mainfrom
pin-hrx-res-scale
Sep 29, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
pin-hrx-res-scale

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Moves third_party/llama.cpp b408831 -> 45c1770 (fork #35).

ZAYA scales both residual branches twice per layer, (x + bx)·sx + (r + br)·sr. On HRX that was up to nine tiny ADD / MUL dispatches per layer; each side is now one dispatch. Measured on strixhalo (gfx1151), ZAYA1-8B Q4_K_M on HRX0:

  • dispatches per token 3,366 -> 3,165;
  • bitwise the same output: 288/288 greedy tokens, max |Δlogprob| 0;
  • decode 67.0 -> ~69.6 tok/s (3 interleaved rounds);
  • pp512 2,109 -> 2,183 tok/s;
  • Qwen3 doesn't match the pattern.

Part of RFC #213 gate 1. docs/hrx.md gets a line on it. The next ZAYA step, skipping no-op CONTs (+2.4%, fork #36), will come as its own pin.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@context7

context7 Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Docs7 for 1bit-monster/engine

Result Status Action
Deployment ➖ Not used —
Content review ➖ Did not run. This site has no agent runs available this month. Wait for the monthly reset or check your Docs7 plan. —

Commit 737e653

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 737e653)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Claim Verification

The PR claims performance improvements such as reduced dispatches (3,366 -> 3,165) and increased decode speed (67.0 -> ~69.6 tok/s) for ZAYA1-8B Q4_K_M on HRX0. These claims must be verified with actual measurements on the specified hardware (strixhalo gfx1151) to ensure they are not overstated or based on incorrect assumptions.

- **Fused residual scale (llama.cpp #35).** ZAYA scales both residual branches twice per layer,
  `(x + bx) * sx + (r + br) * sr`, which ran as up to nine ADD / MUL dispatches per layer. Each side
  now runs as one dispatch (`common.res_scale_pair.*`, `ops/res_scale_pair_f32.loom`): 3,366 -> 3,165
  dispatches per token, bitwise the same output, ZAYA1-8B HRX0 decode 67.0 -> ~69.6 tok/s and
  pp512 2,109 -> 2,183. `GGML_HRX_DISABLE_DISPATCH=res_scale_pair` turns it off. At decode most of
  a ZAYA token is small dispatches and the ~1.8 us gaps between them, not matmuls; the profiler
  (`HRX_PROFILE_MODE=dispatch`) shows which ones. Profile through `llama-server`: `llama-bench`
  under the profiler fails.
Version Pin Consistency

The version pin for llama.cpp (hrx) has been updated from b408831 to 45c1770. While this is consistent with the PR description, it's important to confirm that this change aligns with the intended behavior and that all related components (like the HRX backend) are compatible with this specific commit.

"llama.cpp (hrx)": "45c17708344627b0731ff28d4075da65a3bfa6ff",

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 8f752cf

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 737e653

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant